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Abstract 

We present the first evolved solutions to a computational task 
within the A/^euronal Organism £^volution model (Norgev) of 
artificial neural network development. These networks dis- 
play a remarkable robustness to external noise sources, and 
can regrow to functionality when severely damaged. In this 
framework, we evolved a doubling of network functional- 
ity (double-NAND circuit). The network structure of these 
evolved solutions does not follow the logic of human cod- 
ing, and instead more resembles the decentralized dendritic 
connection pattern of more biological networks such as the 
C. elegans brain. 

Introduction 

The complexity of mammalian brains, and the animal be- 
haviors they elicit, continue to amaze and baffle us. Through 
neurobiology, we have an almost complete understanding of 
how a single neuron works, to the point that simulations of 
a few connected neurons can be carried out with high pre- 
cision. However, human designed neural networks have not 
fulfilled the promise of emulating these animal behaviors. 

The problem of designing the neural network structure 
can be generalized to the problem of designing complex 
computer programs because, in a sense, an artificial neu- 
ral network is just a representation of an underlying com- 
puter program. Computer scientists have made substantial 
progress in this area, and routinely create increasingly com- 
plicated codes. However, it is a common experience that 
when these programs are confronted with unexpected situa- 
tions or data, they stall and literally stop in their tracks. This 
is quite different from what happens in biological systems, 
where adequate reactions occur even in the rarest and most 
uncommon circumstances, as well as in noisy and incom- 
pletely known environments. It is for this property that some 
researchers have embraced evolution as a tool for arriving at 
robust computational systems. 

Darwinian evolution not only created systems that can 
withstand small changes in their external conditions and 
survive, but has also enforced functional modularity to en- 
hance a species' evolvability ( Kirschner and Gerhart, 1998| l 
and long-term survival. This modularity is one of the key 



features that is responsible for the evolved system's robust- 
ness: one part may fail, but the rest will continue to work. 
Functional modularity is also associated with component re- 
use and developmental evolution ( |Koza et al., 2003| l. 

The idea of evolving neural networks is not new 
( IKitano, 19901 |Koza and Rice, 1991| >, but has often been 
limited to just adapting the network's structure and weights 
with a bias to specific models (e.g., feed-forward) and us- 
ing homogeneous neuron functions. Less constrained mod- 
els have been proposed ( |Belew, 1993[ [Egg enberg er, 1997| 
IGruau, 1995[|Nolfi and Parisi, 1995| », most of which encom- 
pass some sort of implicit genomic encoding. In particu- 
lar, developmental systems built on artificial chemistries (re- 
viewed in Dittrich et al. 2001) represent the least constrained 
models for structural and functional growth, and thus of- 
fer the possibility of creating modular complex structures. 
Astor and Adami (2000) introduced the Norgev (A^euronal 
Organism Evolution) model, which not only allows for the 
evolution of the developmental mechanism responsible for 
the growth of the neural tissue or artificial brain, but also 
has no a priori model for how the neuron computes or learns. 
This allows neural systems to be created that have the poten- 
tial of evolving developmental robustness as found in nature. 
In this paper, we present evolved neural networks using the 
Norgev model, with inherent robustness and self-repair ca- 
pabilities. 

Description of Norgev 

Norgev is, at heart, a simulation of an artificial wet chem- 
istry capable of complex computation and gene regulation. 
The model defines the tissue substrate as a two-dimensional 
hexagonal grid on which proteins can diffuse through dis- 
crete stepped diffusion equations. On these hexagons, neu- 
ral cells can exist, and carry out actions such as the produc- 
tion of proteins, the creation of new cells, the growth of ax- 
ons, etc. Proteins produced by the cell can be external (dif- 
fusible), internal (confined within the cell and undiffusible) 
or neurotransmitters (which are injected through connected 
axons when the neuron is excited). Cells also produce a con- 
stant rate of cell-tag proteins, which identify them to other 
cells and diffuse across the substrate. 



Each neural cell carries a genome which encodes its be- 
havior. Genomes consist of genes which can be viewed as 
a genetic program that can either be executed (expressed) or 
not, depending on a gene condition (see Fig.[l]). A gene con- 
dition is a combination of several condition atoms, whose 
values in turn depend on local concentrations of proteins. 
The gene condition can be viewed as the upstream regula- 
tory region of the genetic program it is attached to, while 
the atoms can be seen as different binding modules within 
the regulatory region. Each gene is initially active (activa- 
tion level 0=1) and then each condition atom acts one af- 
ter another on 0, modifying it in the [0, 1] range, or totally 
suppressing it (0 = 0). Table[l] shows all the possible condi- 
tion atoms and how they act on the gene expression level 
passed on to them. 



Gene 2 



complete description of the Norgev model and its evolu- 
tion operators (mutation and crossover) can be found in 
( lAstor and Adami, 20 QQ) . 
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Figure 1 : Neural cells are placed into an hexagonal grid and 
then start executing their genome, which consists of a series 
of conditions followed by a series of expression actions. 



Cond. Atom 


Evaluation value 9 


SUP[CrPx] 
NSUP[CTPx] 
ANY[Prx] 
NNY[Prx] 


= 9, if cell is of type CTPx 
= 9, if cell is not of type CTPx 
= 9, if [PTx]^0 
= 9, if [PTx\=^ 


ADD[Prx] 

suB[prx] 

MUL[Prx] 

AND[Prx] 

OR[PTx\ 

NAND[Prx] 

NOR[Prx] 

NOC[prx] 


= RU%+[PTx\) 

= rI{Q-[PTx]) 

= d^lPTx] 

= min{Q,[PTx]) 

= max{Q,[PTx]) 

= l-AND[PTx] 

= \-OR[PTx] 

= 9, the neutral condition 



Table 1: Repressive and evaluative condition atoms: The 
SUP and NSUP conditions evaluate the cell-type of the cell 
in which they are being executed. On the other hand, ANY 
and NNY repress the gene under the influence of any type 
of protein (internal, external, cell-type or neurotransmitter), 
where ' [PTx] ' stands for the concentration of protein PTx. 
The neutral condition is special and acts as a silent place 
holder. Rq{) saturates the activation into the [0,1] range. 

Once a gene activation value has been reached, each 
of the gene's expression atoms are executed. Expression 
atoms can carry out simple actions such as producing a spe- 
cific protein, or they can emulate complex actions such as 
cell division and axon growth. Table |2| contains a com- 
plete list of expression atoms used in Norgev. A more 



Expr. Atom 


Action description 


PRD[X7] 


produces substrate XY 


SFL[CTPx] 


divide, offspring of type CTPx 


GRA[X7] 


grow axon following XY gradient 


GDR[X7] 


grow dendrite following XY gradient 


EXT 


excitory stimulus XY 


INH 


inhibitory stimulus XY 


M0D+[A^7x] 


increase connection weights 


MOD-[NTx] 


decrease connection weights 


RLX[NTx] 


relax weights 


DFN[NTx] 


define cells neurotransmitter 


NOP 


null action, neutrality 



Table 2: Expression atoms. Each is influenced by in a 
different way. Eor PRD it states the production quantity; for 
SPL, GRA and GDR the probability of execution; for EXT 
and INH the stimulus amount; for MOD+, MOD- and RLX 
the increase, decrease and multiply factor; and for DEN and 
NOP, has no influence. 

We know that in cellular biology, gene activation leads to 
the production of a specific protein that subsequently has a 
function of its own, ranging from enzymatic catalysis to the 
docking at other gene regulatory sites. In this model, the 
most basic expression element is the production of proteins 
(local or externally diffusible) through the PRD [PTx] atom. 
These can then interact and modulate the activation of other 
genes in the genome. In this sense, it can be argued that they 
are only regulatory proteins. However, at least abstractly, 
genes in this model need not only represent genes in bio- 
logical cells but can also represent the logic behind enzyme 
interaction and their products. Thus, Norgev 's genome en- 
codes a dynamical system that represents low level biolog- 
ical DNA processes, as well as higher level enzymatic pro- 
cesses including long-range interaction through diffusible 
substances like hormones. However, the objective is not to 
create a complete simulation of an artificial biochemistry, 
and thus other expression atoms are defined that represent 
more complex actions, actions that in real cells would need 
a whole battery of orchestrated protein interactions to be ac- 
complished. 

Organism example 

The best way to understand the model is probably to sit 
down and create by hand a functional organism. Here we 
will present a handwritten organism (Eig.|2|) and explain how 
it develops into a fully connected neural network that com- 
putes a NAND logical function on its two inputs and sends 
the result to its output. 

The organism, which we named Stochastic, relies on the 
random nature of the underlying chemical world to form its 
tissue structure. When an organism is first created, a tissue 
seed (type CPT) is placed in the center of the hexagonal grid. 
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Figure 2: Genome of Stochastic 



two sensor cells on the left of the grid and an actuator cell 
on the right. These then diffuse their marker proteins CPT, 
SPTO, SPTl and APTO respectively. In the first time step, 
only the first gene (Fig. |2|l is active in the tissue seed and 
all the rest are suppressed. This gene will always be active 
and step after step will split off cells of type ACPTO until all 
the surrounding hexagons are occupied by these cells. Af- 
ter that, the seed does not execute any further function other 
than secrete its own cell type protein CPT. The new cells 
will, in turn, also split off more cells of type ACPTO (gene 
5), and so make the tissue grow larger and larger (time=4 in 
Fig.lSj. In a sense, these cells provide a cellular support for 
further development of the actual network, and could thus 
be called glial-typc cells, in analogy to the supportive func- 
tion glial cells have in real brains. These glial cells can split 
off three different types of neurons. If the signal from the 
actuator APTO is greater than the signal from the tissue seed 
CPT, then a neuron of type ACPT3 will split off with prob- 
ability p > (gene 2). On the other hand, if the external 
protein signal of sensor SPTO is strong compared to the ex- 
ternal protein of sensor SPTl, then instead a neuron of type 
ACPTl will split off with p>0 (gene 3). Last of all, if the 
signal SPTl is greater than SPTO, then it is more likely that 
a neuron of type ACPTl will split off (gene 4). This is all 
that these glial cells of type ACPTO do: split off more glial 
cells, or any of three differentiated neuron types depending 
on how close they are to the sensors or the actuators. 

These three cell types {ACPTl, ACPTl and ACPT3), will 
then form the actual neural network that will do all the pro- 
cessing. Through gene 6, cells of type ACPTl will grow a 
dendrite towards sensor SPTO and define their default neu- 
rotransmitter as NTl. In the same way, cells of type ACPTl 
will have gene 7 active and will grow a dendrite towards 
sensor SPTl and define their neurotransmitter as NTl. Last 
of all, gene 8 is active in cells of type ACTP3, and will di- 
rect the growth of dendrites towards cells of type ACPTl and 
ACPTl and an axon towards the actuator APTO. In the end, 
each sensor SPTO and SPTl is connected to every neuronal 
cell ACPTl and ACPTl, and all the ACPT3 neuronal cells 
are connected to the actuator APTO (time=120 in Fig. 13. 
However, which and how many ACPTl and ACPTl neurons 
connect to which and how many of the ACPT3 neurons relies 
on stochastic axonal growth, preferably connecting neurons 
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Figure 3: Successive stages in the developmental growth of 
the Stochastic neural tissue. 



that are nearer on the hexagonal grid. Moreover, all neu- 
rons end up connected after the axonal growth process has 
finished, forming a fully functional NAND implementation. 

We still need to understand how the neurons actually pro- 
cess the signals passing through them. This is mediated 
through genes 9 and 10. Neurons ACPTl and ACPTl act 
as relays of the sensor signals through gene 10. That is, 
whenever they receive any neurotransmitter of type eNT (de- 
fault sensor neurotransmitter) they will become excited and 
inject their gene-defined neurotransmitters through their ax- 
ons. Neuronal cells of type ACPT3 will then compute the 
NAND evaluative action on the amount of neurotransmit- 
ters NTl and NTl injected into their cell bodies and activate 
accordingly (gene 9). Their activity causes the default neu- 
rotransmitter to be injected into the actuator, thus finalizing 
the simulated input-output NAND computation. 

Robustness of Stochastic 

While Stochastic' s neural tissue will always look different 
every time it is grown because of the stochastic nature of 
neuronal splitting, it always forms a processing network that 
correctly computed the NAND function. This confers some 
robustness to the phenotype of the network in spite of the 
stochastic, but genetically directed, growth process. 

However, the developmental process is far more robust 
than that. For example, we can manually kill (remove) neu- 
rons of a fully developed tissue and have a similar functional 
(but somewhat scarred) tissue grow back. Fig|4|shows an ex- 
ample where we even removed the tissue seed CPT, which 
has an important role in the organisms development (without 
its external signal, glial cells of type ACPTO do not prolif- 
erate). While the morphology of the self-repaired tissue has 
changed, it still computes the NAND function. More than 



Figure 4: Robustness of Stochastic under cell death. Half 
the neural tissue from Fig. O was removed (left). After 80 
time steps a different, but functional tissue arises (right). 



anything, this observation helps illustrate the potential capa- 
bilities of developmental processes in artificial chemistries 
to create robust information processing neural tissues even 
under the breakdown of part of their structure. Note that the 
self-repair property of Stochastic was not evolved (or even 
hand-coded), but rather emerged as a property of the devel- 
opmental process. Naturally, these robustness traits can be 
augmented and exploited under suitable evolutionary pres- 
sures. 

Evolution of organisms in Norgev 

Here, we present the evolutionary capabilities of Norgev, 
that is, how its genetic structure and chemistry model al- 
low for the evolution of developmental neural networks that 
solve pre- specified tasks. In the previous section we pre- 
sented the Stochastic organism, which grew into a neural 
tissue that computed a NAND function on its inputs. Our 
goal was to study how difficult it would be to double the tis- 
sue's functionality and compute a double NAND on three in- 
puts, and send the result to two outputs (Fig. 13). Because one 
of the mutational operators used in the Genetic Algorithm is 
gene doubling (see Astor and Adami, 2000), we surmised 
that there was an easy route through duplication and subse- 
quent differentiation. Because of the universality of NAND, 
showing that more complex tissues can evolve from Stochas- 
tic suggests that arbitrary computational tissues can evolve 
in Norgev. 

The input signal was applied for four time steps (the time 
for the input to pass through the tissue and reach the out- 
put), and then the output was evaluated by a reward function 
R=l — {yi {x) — ti (x) Y where x is the input, y the tis- 
sue's output, and t the expected output. Organisms were 
then selected according to a fitness function given by the 
average reward over 400 time steps, and a small pressure 
for small genome sizes and neuron numbers. Mutation rates 
were high and evolution was mainly asexual. Details of the 
experiments will appear elsewhere (Hampton and Adami, in 
preparation). 

We evolved organisms that obtained the double NAND 
functionality in two separate runs on massively parallel clus- 
ter computers, over several weeks. The two solutions were 
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Figure 5: Evolution objective: to double the functionality of 
the original organism. 



very different in both structure and algorithm. The simplest. 
Stochastic A, evolved the fastest with the more straightfor- 
ward morphology (Fig.|6|l. Its genome is short (Fig.Q when 
compared to evolved organisms in other runs, but is substan- 
tially more difficult to understand compared to its ancestor. 




Figure 6: Stochastic A neural tissue expressing 6 different 
cell types. Most of the axonal connections that spread out 
from the central sensor are not utilized. Instead, the actual 
computation takes place in a compact area near the center. 

After careful analysis of the genome, paired with an eval- 
uation of the physical connections present in the neural tis- 
sue, we came to the conclusion that the organism had not 
reused any genomic material to double the NAND function, 
but had instead completely rewritten its code to implement a 
shorter and more efficient algorithm when compared to the 
ancestor we wrote. Let us embark once again in a quick 
step-by-step genome analysis. Gene 1 is active in the tissue 
seed, which then splits off a cell of type ACPTO and APT2. 
After this, the gene is forever shut off because of the repres- 
sive NNY(apt2) condition. Cell ACPTO then splits off cells 
of type ACPTl, ACPT2 and ACPT3 through gene 2. This 
gene is always active, and thus ACPTO cells are always in an 
inhibitive activation state (due to action atom INHl). Gene 6 
makes ACPTl cells grow a dendrite to sensor SPTO and have 
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Figure 7: Genome of evolved Stochastic A organism. Gene numbering is maintained from the ancestral genome, and gene 1 1 
is a new gene which was randomly created. Gene atoms in light gray appear to be useless and are considered "junk". 



same-type daughter cells. These are the cells that cover the 
whole substrate in Fig. |6l Gene 7 causes ACPT2 cells to 
grow a dendrite towards sensor SPTl, an axon towards actu- 
ator APr7 and define its neurotransmitter as NT2. Through 
gene 8, ACPT3 cells grow a dendrite to sensor SPT2, a den- 
drite to cells ACPT2, and an axon to actuator APr7 . Gene 
1 1 is the most cryptic. This gene is only active in the first 
~3 time steps of the organism's life, and effectively makes 
cells of type ACTPO, ACPTl (only the ones in the center, not 
all the rest) and ACPT2 grow an axon towards the actuator 
APTO. Once the tissue has developed, gene 10 is used by all 
cells for processing sensory information (neurotransmitter 
eNT), on which it performs a NOT function. 



gene 10 is used for neural processing once the tissue has de- 
veloped, and it thus has a structure more conducive to further 
function doubling. 
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Figure 8: Effective neural circuit grown by Stochastic A. 
Dashed axonal connections grow due to gene 1 1 , which is 
only active during the first moments of the organisms life. 
Axons and neurons that have no influence on the final com- 
putations are rendered in light gray. 

The effective neural circuit is shown in Fig. [8] The result 
is processed in three time steps instead of the incorrectly 
postulated minimum of four time steps. This is due to an 
implicit OR function computed by the actuator cells that we 
did not anticipate, but which was discovered and exploited 
by the organism. The neural tissue is applying a NOT func- 
tion at a relay of its inputs, and then an OR on the actuators 
to arrive at the double NAND (Fig. |9ll. The resulting sim- 
plicity of the organism is apparent from the fact that only 



Figure 9: The computation carried out by Stochastic A. 

Another organism that solved the problem was Stochas- 
tic B, which took considerably longer to evolve, and that 
turned out to be highly complex and difficult to understand. 
In Fig. [ToJ cellular structures can clearly be seen in which 
stripe-like patterns of two different neural types succeed one 
another. These stripes were different for each organism, and 
reflect a stochastic development. The axonal connections 
linking all these neurons are so interwoven that it is difficult 
to believe that this organism is actually acting on its inputs 
instead of undergoing some recurrent neuronal oscillation. 

We were unable to describe the development and internal 
workings of this organism due to its complexity. However, a 
complete description is in principle always possible because 
of our access to all of the organism's internal state variables, 
and more importantly, to its genetic code: the source of its 
dynamics. Taking the first steps in that direction, we studied 
the neuronal activation under each of the eight possible input 
configurations (Fig.fTol). We can clearly see neuronal activ- 
ity that follows the striped pattern on the right-hand side of 
the tissue (for inputs of the form xOx ^11). Remarkably, the 
left side of the tissue does not follow the same organization 
and thus we theorize that although they have the same cell 
type, they have differentiated internally even further depend- 
ing on their position on the tissue. We came to the conclu- 
sion that this organism is not performing the same internal 
computation as Stochastic A. We can see this by inspect- 
ing input 110 ^ 01, and noticing that no tissue neurons are 
activated, and thus there is no neuron performing the NOT 




Figure 10: Neuronal activity of a Stochastic B neural tissue under the eight possible binary input combinations, where active 
neurons are shaded, and inactive neurons white. This activity only reflects neurotransmitters that will be injected by active 
neurons down their axonal branches. 



function on the last input. 

Conclusions 

Biology baffles us with the development of even seemingly 
simple organisms. We have yet to recreate insect neural 
brains that perform such feats as flight control. As an even 
simpler organism, the flatworm C. elegans, has a nervous 
system which consists of 302 neurons, highly interconnected 
in a specific (and mostly known) pattern, and 52 glial cells, 
but whose exact function we still do not understand. Within 
Norgev, we have shown that such structural biocomplexity 
can arise in silico, with dendritic connection patterns sur- 
prisingly similar to the seemingly random patterns seen in 
C. elegans. And we might have been baffled at the mecha- 
nism of development and function of our in silico neural tis- 
sue if it were not for our ability to probe every single neuron, 
study every neurotransmitter or developmental transcription 
factor, and isolate every part of the system to understand its 
behavior. Thus, we believe that evolving neural networks 
under a developmental paradigm is a promising avenue for 
the creation and understanding of robust and complex com- 
putational systems that, in the future, can serve as the ner- 
vous systems of autonomous robots and rovers. 
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